Street images classification according to COVID-19 risk in Lima, Peru: a convolutional neural networks feasibility analysis

Objectives During the COVID-19 pandemic, convolutional neural networks (CNNs) have been used in clinical medicine (eg, X-rays classification). Whether CNNs could inform the epidemiology of COVID-19 classifying street images according to COVID-19 risk is unknown, yet it could pinpoint high-risk places and relevant features of the built environment. In a feasibility study, we trained CNNs to classify the area surrounding bus stops (Lima, Peru) into moderate or extreme COVID-19 risk. Design CNN analysis based on images from bus stops and the surrounding area. We used transfer learning and updated the output layer of five CNNs: NASNetLarge, InceptionResNetV2, Xception, ResNet152V2 and ResNet101V2. We chose the best performing CNN, which was further tuned. We used GradCam to understand the classification process. Setting Bus stops from Lima, Peru. We used five images per bus stop. Primary and secondary outcome measures Bus stop images were classified according to COVID-19 risk into two labels: moderate or extreme. Results NASNetLarge outperformed the other CNNs except in the recall metric for the moderate label and in the precision metric for the extreme label; the ResNet152V2 performed better in these two metrics (85% vs 76% and 63% vs 60%, respectively). The NASNetLarge was further tuned. The best recall (75%) and F1 score (65%) for the extreme label were reached with data augmentation techniques. Areas close to buildings or with people were often classified as extreme risk. Conclusions This feasibility study showed that CNNs have the potential to classify street images according to levels of COVID-19 risk. In addition to applications in clinical medicine, CNNs and street images could advance the epidemiology of COVID-19 at the population level.

2) Introduction is rather very small and the authors have left out many works that have utilized Google Street Imagery along with ML to classify health risks. (For eg. Nguyen QC, Huang Y, Kumar A, Duan H, Keralis JM, Dwivedi P, Meng HW, Brunisholz KD, Jay J, Javanmardi M, Tasdizen T. Using 164 Million Google Street View Images to Derive Built Environment Predictors of  3) "However, there is no evidence on convolutional neural networks being used for population health" (this is a very strong statement to make and there are many a papers that uses ML approaches (CNN being just one of them)) to solve population health challenges.
4) The selection of just binary label seems to be based on convenience. There is no justification for such a decision. 5) There should be a brief description about how the transport department has classified the images. Then it would be interesting to compare the key features that were used by the department for the classification and the features identified by the Neural Network. 6) Ours is a proof-of-concept work (This is mentioned in multiple places which is unnecessary) 7) "The main indications for a moderate risk classification were the presence of green areas and lack of close nearby buildings ( Figure  2). Areas close to buildings or with a considerable presence of people were often classified as extreme COVID-19 risk. The presence of cars did not seem to impact the classification" This is the main section of the paper and needs to be elaborated. The presence of people could be a key indicator, but this is currently poorly explained and even the Gradcam diagrams doesn't clarify anything concrete. 8) As previously mentioned, as this is a supervised algorithm the details about labelling step is key. If a bustop is classified as extreme are the 5 images related to it classified as extreme. Such details should be provided and its imperative that the labelling strategy should be clearly explained (eventhough it is done by the transport department).

REVIEWER
Taha E. Taha Menoufia University REVIEW RETURNED 26-Jun-2022 GENERAL COMMENTS -Implementation procedures and results have been well-reported -Both the methodology used and flow of ideas are ok -Contribution of this work is good;

Reviewer #1
C1. This paper should focus more on the public health impact rather than the machine learning aspect. Currently the paper seems to be just focused on CNN rather than Covid-19 risk. R1. We have further elaborated on the public health relevance of our work. Please, refer to: • Introduction section (p. 04 of tracked version) which was edited to elaborate on the public health implications of our work in a clearer fashion.
• Methods section, "Rationale" sub-heading (pp. C5. There should be a brief description about how the transport department has classified the images. Then it would be interesting to compare the key features that were used by the department for the classification and the features identified by the Neural Network. R5. Unfortunately, the exact methods or criteria followed by local authorities to classify bus stops are not available. In other words, how they defined a bus stop was at moderate, high, very high or extreme risk, was not provided. This has been acknowledged in the Methods section and discussed in the limitations.
[Methods -p. 06 of tracked manuscript] "Although this is an official source of information from a government branch, details of how the bus stops were classified are not available; please, refer to the discussion section where we further elaborate on this caveat." [Discussion -p. 19 of tracked manuscript] "Nevertheless, we used official information which is provided to the public for their safety and to inform them about the progression of the COVID-19. Because it is an official source of public information, we trust their method for classification is sound and based on the best available evidence. This limitation should not substantially bias our model or results because the labels were clearly available from the data provider (transport authority), and we did not have to make any assumptions nor manual labelling. However, this may limit the external reproducibility of our work because other researchers may not label their images following the same criteria by our data source. We argue that this should not rest importance to our work because which could serve as basis for future research in the area in which the underlying labelling criteria are clearer."

C6. Ours is a proof-of-concept work (This is mentioned in multiple places which is unnecessary)
R6. The manuscript has been edited throughout to remove this statement. The "proof-of-concept" term has been kept in two sentences only: on page 05 (Methods) and on page 12 (Discussion) of the tracked version.
C7. "The main indications for a moderate risk classification were the presence of green areas and lack of close nearby buildings (Figure 2). Areas close to buildings or with a considerable presence of people were often classified as extreme COVID-19 risk. The presence of cars did not seem to impact the classification" This is the main section of the paper and needs to be elaborated. The presence of people could be a key indicator, but this is currently poorly explained and even the Gradcam diagrams doesn't clarify anything concrete.
R7. We further elaborated on the Gradcam analysis in different sections of the revised manuscript.
[ C8. As previously mentioned, as this is a supervised algorithm the details about labelling step is key. If a bustop is classified as extreme are the 5 images related to it classified as extreme. Such details should be provided and its imperative that the labelling strategy should be clearly explained (even though it is done by the transport department).
R8. The fact that it is unknown how exactly each bus stop was labelled by the local transport authority is acknowledged in the Methods and Discussion sections. Please, refer to our fifth answer above for further details on how we have addressed this comment raised by the reviewer.
Whether the surrounding area to the bus stop received the same label as the index bus stop has been detailed in the Methods section (p. 06 tracked version). -Implementation procedures and results have been well-reported -Both the methodology used and flow of ideas are ok -Contribution of this work is good R1. We appreciate the positive feedback by this reviewer. We are glad the reviewer found merit in our work.